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ABSTRACT' ' , # 

This paper describes the development of materials to 
test English language skills and disc usbes some of the procedures and 
rationale which were- employed in that development. The materials were 
designed for use in the New Canadian Study (196 7-6.8) , which was 
concerned’ primarily with the school success of ‘students who lealned 
English as a second language. This document discusses the theory 
behind the development of these testing instruments and- Retails of 
the six-part instrument devised to test aspects of auditory 
perception and 'vocabulary. Each part is considered and described. .A < 
final chapter presents a preliminary evaluation of the materials. 
Shortcomings mentioned -include- a lack of coverage of. grammatical 
structure/ the language of the directions, and the idea of time 
limitations. Future use of the materials is also discussed;. ; 
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FOREWORD 



Mr. Sugunasiri developed and /prepared 
th'e test, materials. This report is based' 
on hi-s extens ive notes ; he had no opportunity 
to review this text. ■ ..." ‘ - 
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' . ■ ..TESTING SOME ENGLISH ’LANGUAGE SKILLS: 7 ■ • . , . V.. ' " 

‘ RATIONALE DEVELOPMENT AMD DESCRIPTION” .. . . ‘ ' ' ■ 
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* 

V 


" CHAPTER I— - INTRODUCTION ... , A 

• '' • ■ * r .. v 

. • ' % . \ ■ • 0 

■ „ ; ' ' H :• . * . .• > 

, . * . . ■ ■■ • ' • ' 1 ; ' ' >1 

•This paper describes the development of materials to test English- 
. ... - _ ..: , . : ; ' . ■ . ■ . . : . ... * 

language skills and provides some of the procedures and .rationale v/'hich . • 7/ ■ 

' : '' ' ' r . . ' 7. % • • . '7 7 

■ were employed. The materials were designed for upe in the_ New Canadian 1 . , 1 


/ 


• Study (1967-63) which was concerned primarily with the school success 1 " of * ■ 


..'o'. 


•. .students who learned • English -as a second language. As a part of th^ Study ' ‘ •. .. 

• .* : it was desirable to use some ..ins truriient to assess the way in which English 

.. . •* ■ V V ‘ ^ . * . ■' * • • ^ ... ;1 •; ’ <!.;■ 

was understood and* used by such students — qualitatively and/or quantitatively. 


> ' 


It soon became apparent that this would not be an easy , task because ' no, existing ■ % 

test seemed suitable* for the purpose. *• 

. One early idea was to' construct a form of vocabulary test . which V 

; would Explore richness and depth of -vocabulary, not jus t in terms of the , V 


.. . • , t 


number of meanings an individual might be -able, to assign to^)a word (e.g.,, 

•-"' ‘ § ‘ # V ^ ‘ ’ V” 

* r 

''strain") *but also in tprms of hi's ability to use and understand this 4 • , 

. ° • ; " 

'• multitude of meanings in. context (e.g,, "The qifality'of mercy is not 

* • • ^ . 


■ t r 


strain'd.,..'"). It is recognised tha-t abstract words such as "quality"’ f • • • 

ahd "mercy" vary* in .meaning from culture to culture. ThuSj to the.' native - • # 

... u- _ . ^ 

speaker of English who' sees them in the above context, "quality" and "mercy" • ' *’ 

have. a<great numbervof loadings or referents peculiar not only to the way ' ' 


n 

• ' 

■ ►* 


in/’which thej f are juxtaposed in this particular sentence but also peculiar L 

to his cultural heritage. Perhaps/ specific reference might even be made in ' " 

/ - \ / * • » 

> a discuss i6n 6f this short sentence N to North American or even Ontario usage,' 

r** «* / * 


t» '-V 


at least by the speaker to whom such usage is non-cons cio us ly natural;*. ..'The 

. - • t . J' * . ‘■'■X* ■ 


it ... . 

0 , 
ERLC 


. * ‘ . ‘ - * • ; / ■. . ' : " • - • - • . *• : 

/ . . v 

» > . ^ /A . ■ j 

1 * ' /•% ’ r. ** / * • . * ■ . * - l; \ 

: . • - ' / . • K - , • . . * 

•. # . / • • t-/ , . ■ * ' >'rXv. - 

' 9 • / f v . ■' .* • . v ' ' 



extent to which' a New Canadian was aware of and could use these loadings’’ 

v ■■ ■■ . . : * . . ” ' . • y 

would be a good indication of his acculturation 'as well as of his. ability 
: • * * . ' • - • * » , 

) + ■' • • • . ' • 

/in English . ■ • • ; . * ‘ 

* Shch a test would likely be limited’ in usefulness, "unfair," -and 

■ ; ' •/. . . ,\"v- ■■■;■;. • 4 : ■■ . ‘ ’ ^ 

extraordinarily difficult to construct and validate. The nooulution of the 



New Canadian Study was to include a representative sample of students chosen 

j.’ • "* • * 

from Grades 5, 7 and 9 -- including students who had recently arrived in *• ■ 

Canada, students who were born in Canada but whose parents tfpoke another 
language, and students who were English’ speaking- monolinguals . ; Because of ' > 
this variety, one P immediate problem was vocabulary, i . e . , the vocabulary of - V 
some of., the students might be far., below the level' of vocabulary tufbd in V ■ ' ' 

the test.. . Thus, no ’differences in degree could be ascertained .among such • 

students below a certain level. This problem arose not only in .considering 

• . • • t •* .. , J . . r 

construction of a "language-in-culturef test, but a^-so in considering 

s» . ’ - , * • ’ _ ’ ’ . 

other tests already in existence. ’ ’ • . v 

». .’Another difficulty with, a vocabulary test is/ that vocabulary and 

intelligence are closely related. While both above and below average children 

speak their native tongues flu’ently, the above average child usually has a 

more extensive vocabulary. Thus; a straight vocabulary test is onenthat 
• * % 
tests, intelligence as much as, or perhaps more than, anything else'. In 

this part of the Study it was planned - to try, as far as possible, to avoid 

testing intelligence. . , . ■’ , 

English "Competence 11 * \ ... 

* . - ’ J ■ ' - 

Such considerations led finally to the linguistic copceptof "competence!' 

To understand the concept of competence, it is first necessary to. understand 

that a 'language is a finite system. Gleason (1961, p. 50 ) has counted 46 • , 

meaningful' sounds (.phonemes) in English. Paul Roberts. (1956; pp.29S -299) notes 



U . ' I '• * •. 1 • <f 

that , there ‘ are seven ba3ic (structural) patterns of English. v-Thur, it can ' ■ 

• . •* , . ■ . 
b$ said that English is a finite systeftsr."' 'A cjailcl reaches linguistic adulthood V 

when he has mastered all the finite" sets of linguistic .elements . ‘The child- 

can then. manipulate the known finite sets of . linguist Lc elements to produde . 

Ian infinity number of utterances that may never have been said earlier, by- . 

I v 1 :*'* ' \ ; : . 

himself or anyone else. This is as true' of EngliMii.^s of any other language'. \ 

A native speaker (or fer that matter anyone who has mastered English as .a. 

V • , ' r -• ; ... 1! • *Y *' ■ 

second language) .can continue producing ..an infinite number of new utterances.-** 1 

\ '■'■■■■ • > ■’. . ^ ' " ... '■ . 

This is f the process .of creative encoding. Other native sneakers who hear . «• . 

_‘.i : *• « . . '• -. *• ■■■• ■. 

sulh uttei 



Jh utterances accept thefti as’; utterances • of their own language;. ..this is • 

\ ' » . . • • • . • N- ■ t 

creative decoding. This .creative aspect of language, both productivity and 

\ . ‘ ‘ ■ ' :/ '*> ' ; / ' . ' ' " 

receptivity, -can be called the native speaker's <JompeU*nce. 

: . - V ■- " ,■ ••'■■■ ' > . - , . '-V'Vv': :• * ■ v 

\ i TJiere is no-aspect of language which is not creative if we* take 

- : V - J ■ ■ - ' ; ' ■ ;• • V- 

an extended view of creativity. We cannot be certain' Whether a child's' 

* v - 1 . ’ : . : : 

.first speech- is creative or imitative, but, as the child attains ■'linguistic 
■-'« - *.. \ - ■ .’••••; * ■ b 

adulthood, he is certainly using his knowledge of- his language, even when he 

■ ■ \ . . ' ’ • ,• 
produces utterances that he has heard. No two situations are identical and. 

each situation, which by definition is new,' demands novel patterns of linguistic 

• V ; • \ ■ ■ 

communicatiion-’where the language used can hardly be" imitative. It is a 
ceaseless process , of ' creating the "unknown" fr<?m known el-emerlts . Seen this 



\ 



V 



way, language is always creative. 

\ * •/ • * \ • • . _ ; 

At this; point a ..distinction must be drawn between competence and 

X 1 ‘ ' ... 

performance . At any given time-point the human organism has stored up a - vast ■ 

-* • ’ * j • t A 

quantity- of information, about everything around him, 7 including language. ' It 



<\ 



l, , 



would never be possible, even. for tlii most insightful investigator , -to find 

. . • \ * ■ ■ i - 

out exactly how^much a berson knows of a given s'ubject, area. A student's 

' - ' ■ ■ A- - / ; ' 

score of 100% in mathematics tells us only thq/t he is .capable of performing 



I 



- 4 



. . \ . 



v ■’* 



all the mathematical calculations about which he has been ouestioned. It 



does, not tell us how much more he is capable of doing. Dor does the "score 

* ■ ; • . . ; .* . ■ • ■ ■ ■ ■ • ' 

q £ less than 100/4. take into consideration other related -.factors- such as 

memory or emotional mood which determine a. .person's behaviour Just as 
much as knowledge does. Thus, it seems, useful to. distinguish .two aspects 

' . i " - - ■' 

of -a' person's linguistic behaviour: his total knowledge and t.he expression > 

v: ■ ■ •' ; v ’ ... 

of that knowledge in a ‘given situation,.- the former, Chomsky calls competence . 

• , ■ • 

the latter performance (Lyons and wales, . 1906, -op. .*1 37-133.) . Since’ the 

' ‘ : ; : -V-."'*' *-.» 

number of utterances a person is capable of 'producing- -is infinite, but tjhe' . f 
actual number he produces at a given time is finite, performance fails far 
short of competence . v • . .. ’ . . . | 

This ^introduction is being written after the ma tor iats were developed 
and used. ' Briefly it summarizes J.he goal that was Set-, an unobtainable goal.' 

- ■ ' . t ■ ■' ■■ • . .. . v ' ■ ; 

The initial posture during...the planning stages was to accept no conceptual 

* ' • ’ ' • ■ 

limitations but rather to devise instruments which could adequately sample 

•a person's knowledge of the "English language. A -useful label for such a set 

of iri§£njments was an "English ..Competence Test" (E.C .T.) . The label, remained 

long after -it was apparent- that the materials developed in a few -months of 

•I’.'. _ "V 

concerted effort were tapping only some aspects of performance and were thus 

v , > • 

not even close to sampling competence. ' ' 

✓ 

The next section briefly presents some of the- ideasiand strategies 
which were considered during the early. dis'cuss ion stages. This is followed 
by a detailed description of procedures and rationale underlying the specific . 
sub-tests. The concluding chapter discusses some' of ' the limitations of this 

. . . . -s' • * ■ ^ . 

material . ' ' J 

. ■* , 

u , K 

As will be seen,' the word "competence" shaulgl now be stricken 
from the title of these sub-tes,ts. Following the .analogy from mathematics, 
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CHAPTER II -- LANGUAGE TESTING 

t 

* * 4 



. • . * 'The following section discusses some of the principles underlying 

'• V ' • , ... • ..... . • •. . 

. * . 4 

language testing. These will put in perspective some of t,he principles . 
underlying thettievelopment of 'llhe English Competence Test (E.C.T. ) 



s 



What to Test 






. \ ■ . 

The first ' question is . '"What do we test?" when assessing* English' 

pv, a second language;. Dr. Finocchiaro, the distinguished* American’ scholar 

. \ V , . ‘ V , ' 

■>f English as a second' language, give3\the* answer: \ 

. : •; ; V’\. .. '. •*• *A % ; •. \ ■ 

v * " .we should, test everything we consider 

. \ important enough in language learning 
\ .to teach-. " • '• . 



7 



(Finocchiaro, 1'967, p.. Ill) y 



Components: Isolated or Integrated? c ... ' 

\V ■ - " ( ■ ' •; • • . ■ . " • - : ■ 

/ "■ English, like any other , language , is made up of a number of • • 

, ’ " * ■#.,■ . . . . • * • *’ *'■];' '■ 

components. It is these that should be tested in •& 'learner of English 

.as a second language. But since speakers do not use these component 'parts 

in isolation, but rather integrate them in speaking, the question arises: 

/ ' f 

do w£~test the component parts', separately or in combination? v 

.. -Lado has this to suy on the subject: ’ , 






"...all we hope to find’ out in a foreign 
language test; is how well he [i.e. the 
learner] has mastered the elements .of the 
system .of signals that constitutes' that 
foreign language."* 

• * ' v (Lado, 1950, p. 77)" 



d 






<■ • 



1 ' Elsewhere, in his book Lah^ua^e 'Testing La do '.’further • clarifies : • 

■’ =' ■ \ - ; . - >\ 

. . • < • ii 



/ 



\ % 



. .whether we test the elements ;of language 
; separately or : as part of one. of these'five ‘ 

' skills [.listening, reading, speaking,' wri ting,' 
an£ translation] we are still testing language. 

The .decision be tween testing , skills or testing' " 

• elements . .- .will be nail;: on- the basis of what 
■ •' infprm^tion x&. need Concerhing .'the., c: indent's 

» . ’ : knowledge pf the grqal Language." ,' i ' 

;* * ' . a ■ ‘ \ , l • (t«ndo, '1961, n.: tl04). ' . | 

_ In the present S ituation', a ". ..•< f all'-st^le resear eh project ■ 
involving New Canadians. . .''.’’.'had been renuested by the’ Board. Although 
the ter.m "New Canadian" is being defined In several ways, for purposes of 
analysis,' it was for this part pf .the -Study considered primarily , in terms 
of ^language. . The Background -Information Questionnaire .asked , "Before 

learning- English did you' leaj-n. any other language?" The subject' who 

* . * \ # , 
answered "yes" ‘to this question would be considered a- ".'few Canadian." 

, In this category, -then, were children whose acquisition of English 

varied anywhere from zero (np.' English) to a hundred per cent (complete mastery) 

Because the Information to be sought from the population concerned the 

t q . *• ' : _ ’ ■ N . 4 . " V >v’ v 

extent to' which English linguistic habits ‘had* been acquired, an instrument 

‘ 

was required which could be used with most, if not all, Hew Canadians. A * . . 

* - ' . K. # - . ■ ■ 

Jokst'xtf production (i.e. speaking)' would not be usable with the* population . 

• • V ‘ « 

because of the existence of those subjects with zero or near zero ability ■ 

in English. There would be other subjects in the. population who spoke 
English', but poorly — strongly accented, for. example, or inadequately, 



as in the case of the person vjho says <1 go>' instead of <I'm going>. 



An 



1 At this point ,it£ls necessary to, introduce the reader to certain symbols ‘ 
which are used* throughout this paper. The written word "through" does not ' 

• exactly correspond to its pronunciation: "oligh" here is pronounced as in. 

'■'boo\" If the exact pronunciation is shown ip- writing (by using a special 
alphabet whj.ch uses mostly Roman letters) then we have a phonemic transcription 
but when it is .writt^d with the regular spelling, then, we have, a graphemic 
transcription. In this .paper, the graphemic transcription of a spoken word(s) 
will be written iwithin_ wedge- brackets '■•(< >)«. as follows: <true> ., • The phonemic 

." transcription will written' within slanted lines (/ /), as follows: ’/true/...; 



instrument of integrated skills , would only -tell us that he did not speak ' 
English well; it would "not tell us how far - he had proceeded' along the path 
to learning English. , . \. .• ” .• v 

. . * • • ' t . * . * 

■ . ■ * . ^ 

; There was . yet another problem to. be considered.- In the 'case pf* * 

a baby, learning to speak- the mother tongue, it has been reported that ". .-.there 

. . y • . . * -o' \ 

is apt tO'tSTra lp; of 2 to 7. months from first learning to utterance" (Penfield * 

and Roberts, p. 243) • It is probable that ip the case of a. second language 

learner as. well there would be a marked time lag between learning and utterance . 

'Thus, there is- no way of. knowing to what extent the utterance <1 go> in the 
: '• ‘ " 4 . ! ■ ' . .. e 

above example %s an indicator of the speaker's knowledge of English. It may 
^ 

even be true of the New Canadian who sfpeak^.iio English at all that h’e has 

1 s s •y _ s. x;. r* v ■ 

"knowledge 11 of English but-*that the first break-through in the nerve p^ths 

# • ■ #l ♦ 
-Reading to articulation have not taken , place; -This position is incidentally 

congruent with Chomsky’s distinction between* linguistic competence and 

' ' " - ^ S C* '/ , ' , ' ' 4 

performance -- that the latter ^ being behavioural, falls short of^ competence " . 
i_/e. a person's knowledge. • i ; / . 

. V ' . „ Even if a test of production, or performance, had been deemed 
desirable, there was also the obvious, practical problem of testing production. 



'u 



It would indeed be an enormous, almost insurmountable task, to. record and 
analyze- the. .speech of some 6000 students who would \ons-titu^e the -study 



population. 
Ideational Fluency 






There was another possibility 



% 

in addition to, what has already been 



described — a test of what flight be<rca!.led ideati^nal^fl^uency. Was^there any 

Way of testing the ."creative aspects of language usage? * Ideatior^t^fluen 'must 

•* *» * • * • 



•be an extension of linguistic competence; ..therefore why not include a test of^ 

V. ',•'** **•' • . . f ■ ’ * 

• ideational flji6ncy? 1 A K 



f 



* V • • 

2 *See McNeil (1968) for a further discussion on this"'. 

« - V . 



* » 
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A 



If a child's- (age 4 or above) speech is compared with ' that '.of 
an adult, one '^f the differences to be found is in the richness of expression . 

Q . ^ ‘f . ‘ ' o ' ' 

of ideas . -Beoause the adult knpws more words, he is likely to express V 

- ...••• * - , • ' ■ / ..." 

an idea better (e.gi, more explicitly). 

. . . • ' * ’ . • • ' i 

The -New Canadian, since he already speaks his own language, J 

* r . . . • -• - 

’ certairily.- has" sqme ideational .fluency. For example, the neurological circuitry 

pertaining to the coticept »<th'e horse is running> iS already present with 
-• / ' * ' t 

reference to his own language J /The problem fo'r the New Canadian is to transfer 

that idea (concept) into the’ sound units of ‘English. (If* he lacks a concept 
, * * 

then he must learn both the concept arid the appropriate sound units of 

■ t • ■ % 0 ‘ 

English.) . ’ • ' ' • ... . .*• “ *• , 

. A. suitable test of ideational fluency would aVbest'tel^F'us whether 

the student has knowledge of a particular concept,, rather than whether he is 

£ . . * ' . • . ' 

able tP„express o^hear it in English! ' In other words, the failure on*th£ 

. 1 V ' ' . . / . , . 

part of a New Canadian onAS/test .of ideational fluency in English mi^hx be 

■ 'v Y - -■ .■ . 

clue to either his' lack of English competence, or lack of ideational competence, 

l 

or both. We would not'know which. If he' were Vested in his own language," 

• . v. • . ■ • ; ... . > j 

there would be no direct indication, of English competence. 

, * C ' * V r . 

Reading and Writing j ; ’• 

Writing is not synonymous with language: it is bnly a representation 

■ 1. Y V ' . / , .? : ■. 1 

-of language by means of symbols. Writing is one step removed from oral 
language. It isv essentially a matter 'of tracing symbols representing the 
sounds" of language. 'But to trace them, one first must have the sound units. 
'Thus, writing is an extension of language;. or writing is a secondary (not in 

importance, but developmen.tally) language ability.. So is reading. Before 

*• "\ " ' , . ' ; . ' ■ ' ■" • ' / 

■ _Li * • .• 

3 . 'Compare in this connection a creative writer and an ordinary adult. Since, 
the creative artist handles language .with more dexterity, there would be 
more richness of express ion 'in his writing than that of .the non-writer . 
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one can read (and understand), one must "know" the' ‘language. If Writing 

i • • f , * 

is tracing symbols , '‘reading is meaningfully vocalising the symbols *8r 
• ' • • ^ 

# ' s. \* 

units of symbols, representing sound. . 

, * » 

• * * • . . ' 4 * " 

The man who reads English knows his Roman alphabet. French is 



also written in the same Roman alphabet, but the English-reading man 



V 



perhaps cannot read French. ,He could pronounce each sound, or. combinations 
of sounds (i:e. words) without understanding any of what he was vocalizing 

•i 

% . I 

(or seeing) . Similarly "there is the case of the ilew Canadian child who 

f | J'i v ■ i f * / 

,, reads , . , .'‘>Bag(y.3h out loud from the page but is unable to tell what he has - 

» • * **’* • • * ‘ 

read. Both cases illustrate that reading, like writing, is different from 

* .... 

‘oral language.. ■ .* . - 



How to Test 



Basing Test on 5j6o Words of Highest Frequency j 

*A further ' limitation was determined by the nature of the study' 

• * * 
population but had a theoretical basis as well. . f . 

* t • 

. . Vocabulary is a component of language that continues .to" enlarge. 

The notion that language is synonymous w^/th vocabulary is therefore a 

t * .... 

misconception. (A Canadian'child with a very^low vocabulary can indeed 
speak English as fluently as an adult, if fluency is defined only as the 

V’ 

ability to manipulate the components of the’ language.) Jo limit the , 



^n^luence of the vocabulary component on the sub-tests it was decided that 
the materials should be constructed -(as. far as possible) using. only the' 



i.e. those words that have primarily a grammatical, or functional , meaning.* 



,.C 






\ ..\.i. 



t 



• j 






Lorge-Thorndike list of the most frequently used 500 words. . ( 

• J t • 

In this.'-list are contained nearly all thfe functors of English, 



To understand this concept, take the word“"<is>. in the utterance <Mom is going>. 



/\ 
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/ . 



y . I 



The only way. we can give it meaning is by referring to its function in an** 

1 ' . . • ' • • V 

utterance. Thus <i r s> denotes the present time of' an action with reference 

/ “ • ' ! . ‘ , /• 

to- one doer. A grammarian wo.uld say that it denotes the present tense . 

.'singular. ' Further we can’ say that, it occurs' before the* * doer in. the statement. 

... . - . . T\ > 

• • 

but again t this is contextual.. So we.say <is> has. only a functional meaning, 
and is therefore a functor ^. Words, with ho n -grammatical meaning, e..g., <mom>- 
are called contentives’. ■* -* 

; ‘ . . “ : " ' • * . V* vV • 

An analysis of the language used by a child liged four who is 

\ * 

/ I # . 

said to be a linguistic adult but whose Vocabulary is nowhere near that- 
of an average adult would. show that the proportion * of -funnftm-s used -by him 
is higher than the proportion of -contentives used v .Partly' this is because. * * 
the functors _arp quite limited in number — Finocchiaro ^ ( 1 96,4 , p. 27) counts*' 1 54 , 
whereas the number of contentives ilmany times larger. It stands to 'reason, 
that the New Canadian learning English as' a second language will have to ' ■ ' 

V ' * . ' * 

master theAinctors in order to be able .to manipulate the language (i.e. be 
fluent in terms of the limited definition) as vocabulary expands the number 

of contentives increases j To have mpre verbfil labels, . however, does not 

. . • < * / . 

ensure an in^eased ability tcf manipulate them, use them p.n sentences, phrases, 
or in the past and prese'nt tenses. .Therefore it seems rpjst appropriate' ■ ■ ’ ’ 

to use the’ word list for the’ purposes which have been outlined. ' 

Furthermore, the contentives included in the first 500. words 4 were 

I • * . ' r . 

.basic enough to be used with ( New Canadians having limited exposure to English, 
thus extending the usability off the E.C.T. , * * 

1 » •’ f ^ . • 

4- Hock^tt (1958, p. 264) prefers functors and contentives -to^ functlon 

• words and context words . • s ince both categories, include forms that are* j 

not words, e.g., <-ing> (functor) . He lists four types of functors:. * j 

1 . substitutes; ^ ‘ .examples: ^ <all>, -<it>, <the> . 

2 1 ^markers; ’ examples: <and>, <in>' . , f 

j " "3- /inflectional affixes; ' examples: <-s> plural, <^-ed> past tense ) h 
' ■ 4. derivatiorial affixes; examples: <-y> e.g., milky;, 

• <-ness> e.g.j happiness. • .* f 



. / . 






•" I 
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. • * * • . / ■ ; v ; * ... 

There can be two criticisms qgainst using -the Lorge-Thorndike 

. . - *' > ' ’• 

•' ^ ■ * v Vi * • * 

list as, a basis for a -discussion ort language: s its basis is the printed ' 

1 ’ 41 - ^ ^ * 

word and not children's speech; the count was taken 24 yfears ago, and as 

such is not representative of today's usage.- 

t . • .* # . ' • ' * 

* (.a 

. The first objection can be overruled by saying that printed ^ 

•> .... •- . * 

* * * 1 • **" 

material may not be far removeq} from standard usage, and certainly 'not to 

. ' " i ’ • ** 

• ^ • * ■ * k. 

such an extent as to be, non-representative of speech. Three reasons can 

•» . * • * 
r * ■ % * . 

be advanced on the second couilt: (a) the first 500 words of any given < 

♦ • " 1 i 

time would likely include* all the functors; (b) there would probably be 

. a large enough number of words in the list that \ ik> representative of the 

• current us^ge; (c) time is more -likely to increass-the usage of Certain 

, contentives, e.g., "atomic" than -functors., 

% ' 

* # t ^ 9 g 

, U^ing the first 500 Word,s i*s also justified as -New Canadians' 

• * , * 

ideational fluency, *.or reading and 1 writ in (^ability, was not being tested. 






Outcomes 

r - 






• * t 

An early decision was made not. to use standardized reading tests. * 

{ . ... ", ... 

In addition a '-picture* vocabulary test wns selected as a component of the 

test battery. ,It appeared that -any. material needed to assess .English 

I i ' . ' - £? 

language skills might best be built. Although sych instruments* .would provide 

* . ,. l # » * '■ 

* ; \ ^ 

' J only coarse scales* -they wovild shift the f opus ‘from classrobm performance to 



language performance - .. Rapidlyjw attempt was made to define operationally 

* ’ ' ' '* r • <* ■ -L 

some bf the component English language skills^nd construct items that would 
* f . V , ’ " ‘ ' • - ' 



require these - skills' for a *correc.t answer. The following chapterr details . 



the steps in constructing these sub-tests. • 

'j 




*>" \ 
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CHAPTER III' — DEVELOPMENT OF THE ENGLISH COMPETENCE fEST 



- This chapter uses the label which by .now had. stuck 'as ‘a term ■ 

to describe the test materials (indeed even the answer sheet was labelled 

' * . • i 

"English Competence").. The fact that this libel cqn now be clearly seen : 



, as inappropriate hopefully will, not turn the reader away from 3 consideration* 

. * - 

of the„ details regarding content selection and Item development. * . : 

\ Format . of tlVe Ten t * . ' 

* • • * * 

\ ♦ n * « * / .$ * 

■*** * ihe test dqnsisted, of six parts y divisible into two abroad Categories; 

auditory, and paper and pencil. • • . \. • . 

_ ' * • ' 1'-. 

, x * ■ • f 

-Parts I to III- tested sspectp of auditory perception'*, and Were 
pre-recorded on tape. The directions to the ^tudeats , the practice examples, • 
r and' the test items -themselves' were ( al3o taped in English'. „ >arts IV to VI. 

. •• ^ t 

tested vocabulary aspects of language and were, in prihted form.' All answers 

•' J /A- 

.were recorded on 'a separate ,Digit.ek answer sheet. Students also ljad a practice 4 

sheet to use with the instructions for each 'part of ‘the test. - ; •' 

N *• , s ' 

f K total running time of the- .taped materials, including directions 

and test itemsy^was 36 minutes'# The>'time limits on the remaining parts 



4 . 

were 5, 3, 'and 4 minutes respectively, exclusive of time- for direction^ 



. (' 



* l 



or 



Parts I and II 



■> . The purpose of Parts I and II was to test two basic elements of 

# ' ' w ■ • 

language.-- sound discrimination and sound recognition : Given two (or more) ’ 

speech .sounds or sequences , of sound, 'p competent speaker is able to 'identify 

whether the two sounds aVe the^lame or different. TbTs is discrimination .- ' 

Sound recognition refers to the abpity of a spedkrer to identify a- speefch ' 

• * • / * ' ’ . ' '• 
sound, or sequence, of sounds ,/us-injg an’fexisting referrent which will probably 



1 



»' 









f 






/ ' 



V 



u - 



include information about both sound .and meaning. Because different Sound:: 



\ \ # * . , , 
and combinations of sounds are significant in different languages, recognition 



\ 



and discrimination -of the sound?' in English' are an important ‘aspect* of 






English competence. Probably in practical terms there Is little difference 

I ' . • "• ' •' ' \ . v. 

in difficulty between recognition and discrimination. 



' *. Part I consisted of. 45 minimal oairs. A minimal pair is two words 

' • '' - ■ . : •/ \ t 

f? , ■ V . ! ■ \ 

that '.sound alike except for one phonemic' difference, e!%. , <5pit:bit>i. The 






* N 



twa- stimulus words were said 'with an interval of approximately* ohe. second 

” • ' & ' , • * * * ' 

• % 

between, them. All items were presented by the same voice -(female adult). 

, , • * - • ,t t 

4 * I 

The subject- was given 4 seconds to decide whether the two words were the’ same 

^ * * 

or -different, t and to mark his answer &y -darkening S (for same) or"D (for - 

4 \ % • '' ft * * 

different) against, the item^on Jihe. Answer Sheet. , - ' ’ * 

Example* 1 — coat . coat' ' SQ Dt) '. ( | p , 

Example 2 — bag ba,ck Sfl DO . ' • 

r l * *• . 

Part- II r also included' 45 items .and .tested sound recognition. The ' 
.voice (adulWemale) first presented one word. Then, .approximately 1 2 seconds 






later, two more words were presented' by the same voice, one of which is the 

* , . 

•• ;<•. ** ■ *. v „■ v • 

same as tjtfe first word uttered. The subject was to #< indicate, by darkening 



A or'B on the Answer Sheet agalnstlthe particular item,_ which of the latter 



two words i^ the sdme as the first 



* 

. A 



> 



Th6 same voice used in Part I presented* all 45 i terns. - 






A 



Example I — poor 
Example 2 — tick- 



4 



power * * . '.poor . Afl '*Bt) 
. / 

. tick- Dick }.•' All BQ 

'•V* / > ... 



5* As a response technique, Lado (1961, p. 54)' favours’ .marking the 
choices that are the same rather than those that ai-e different because 
it approaches slightly -closer the operation of .Language. The listener 
identifies what a thing is first, and then is dble^to tell what it is not." 



* f : 
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1R 



1 

I 



Basis of Item Selection for- Parts I and II 



Gleasorr's list (1961, p. 50)^ of 24 English., consonants 0 and niti'e 



vowels served “as the basis, for the test items. 



? 



Minimal ‘ pairs were iihen^&ought based on .the following criteria: 
1.. -voiced/voiceless contrast ” 



1 - Consonants 



e,xample: /p/: /b/ 



■ -j 



■l 



2. -contrast in initial and final sounds 
* > ' * 
J k 

example (a) .pint bin - 

• Jb) lapr.it\b < * 1 

'■ i ■' •*. . 



3 • 



contrast in^poipts of articulation, 
in' nasaJj consonants (only^ ’’ . *• 

* . f . & 






I ’ Example: * /m/: /n/ 



II - Vowels 



front/back -dimension;, 
"example. / i/ : /u/ f~ 

, / >v 



g 



6. 

‘7 



'8 



See Appendix I for the list. - .. . 

, • ‘ # ’ * • . ,? 
Consonant? -ure measurable or describable on three criteria: voiced/voiceless' 

point of tongue-contact, and degree of .cbnstriction (i.e. whether lull g air • 
is ' completely checked and then released - /fc>/*/b/ etc ;. , . - or is partially ^ 
■(constricted -/f//s/ etc. 1 ). All . nasal, . consonants are voiced and produced 
with total constriction, so that the only contrast is with reference to .the 
point of articulation. . » 7 

r *, . 

"Three 1 variable^ in the positions of tjie( vocal organs are particularly 
significant in the phonetic description -of English vowels. 'The most /' 

important is the .position of the highest part of the tongue. This varies 
.in two dimensions. I1> may be relatively high, mid, or low. It may also 
pe^relsftively- front', central, or baVk.** Note the difference in meaning 
{between central (intermediate -between front and back) and mid (intermediate . 
between high and low). These two variables provide symmetrical charting 
of the -English vowel phonemes:’. FRONT ‘GENlRAL. BACK 

’ ‘ . • ■ ' HIGtf 

' ' ' . «... 'MID 

LOW 



i 

e 

ae 



£ 

i e 

a 



u 

o' 

o 



'The third variable which is of importance in English vowel^f is the round- 
ing of the lips. In'/u/ there is always 'moderate rounding.. In /o/ there •• 

* is usually somewhat less, ‘but always enough to be noticeable . In /•>/ the 
rouncling is still weaker.' In same pronunciations .it may . be so slight as 
to be hardly noticeable, or even lacking. The front and central vowels 
are ne.ver rounded." * (Gleason, 1961,’ pp. 35-36) ^‘ w Not used in test materials 
•*as« native speakers maintained that it does not *o*cpur in Canadian English. 



—m 



T 



<• , v 

$ 

/ 



# ■ • ' . 



•- 16 ^- 

,~7'' 



> 



./ 



2. high/lou' dimension.- 
exampl e . /i/-: ' /e/ 

•' I . \ 

N 3- initial, medial and final* 
example '(a) it: eat * 

, ' . (b) bat: but •*’, 



-p 

(c) way: we ^ 



V 



\ 



4* length ° ’ • • 

♦ •*'* 

example (a)-* bi‘t: beat . • 

Cb) coair: caught , 

5.' nuclei (glides) /nuclei'? 
example bite: bout’ 

4 * • < ^ % ' . * * 

• < 6.' ^ucl.ei/s ingle vowel • 






/ » 

« *• 



example but: bout 
. *\<^^ * 




1 / Il![ QtKer Contrasts : There/ape' other/ significant contrasts 

' . . ,* - which wbuW not) be* covered by the above. 

' l* ' ^ ^ * An Qt.t.omnh. unc /rnnHo •hVwa'Aia. 



^ / 



I ' 

1 



An' attempt, was <made to include the’Se, too, 

' oUoit + -i ,r Jl-.. > ^ 



\ 



,• ‘albeit notv’cpmpfehe.ns.ively 

f •* 1 . ' * ' .IT ' 

‘example: (a) lamp:' ramp t .* 



j * 



(tJ). art: hea &t 



12 . 



a 

4 



■ ; c v.. 



thing: "think^j. fA 



\ 



■f. . 



veal : whebl 



T 

<■ x 



9 Vov.il + semi-^vowel = nuclei. See-Gleason 1961 (pp., 34-35) ‘for a list. 



10 Cl asters* (e . g. , str- as in "street" ); should also have been included 



i . the list, but have not been. . This is a shortcoming. 

% 

. . * s • .1 ' 

11 Example 'from Perren (1967, p. 2/J l 

° * . * . ; ■ . •• ‘ . \ ’ 

12 . These e*xamples, caiin’e from the author'’ s persqnal experience in teaching 

* .English as a second language. ’ ,*/• • . * . ■ '.* 
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mm 
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Once meaningful minimal pairs were found to satisfy the above 



14 * ' ^ ./• 

conditions , three pairs of words were made for each pair. Two pairs 



consisted of one member of the minimal pair- repeated, and the third wa: 



- \ 



✓ 



f- 



/the minimal pair itself-.. Example: pin : pin * j. 

"* ' ■ 

bin : bin- < 

; •.* ’ . "i 

■ pin : bin . "I 

4 . • . ' . * * . 1 

The order was arbitrary: Again, quite arbitrarily, the two members of . 

the minimal pair were almost always written in the following order: " ■'* . 

y ; voiceless :* voiced, e.g., /p" : b/ 

» , ■ V 

A front: back, e.g., /m * : y/ ; /i : u/ 

• 'short V- : long V, e.g., /i': i/ . ( ' 

* 

Generally, minimal- pairs for the consonants were placed first in the list. 

• • 1 p * » . ~ ‘ g 

to be followed by those for vowels. Other elements tested were placed as 
* . # f 
f - , # * ■ 

they were created. ‘ . , 



A * 



\ 

J 






• , - * y. 

. * ' Th e next step was to e^limingite those pairs which included onef 

‘ • •/ ^ ^ 
member used on more than one occasion. For example <bit> was contrasted 

1 / ^ \ t . * • . ■ . ;; 

with <bet> and with <bat> again. Thus^ there would have been two minimal 



^ * 



> 4 






M*- 



'pairs which included <bit>. 

ft 

• This process of elimination left 94 pairs gut of an earlier 102, 

The first 90 pairs were- then! renumbered us in.g'a t table of 'random numbers / , 

• , ’ \ - '* . ' - ]■ * ’ 

This randomization -helped to overcome any bias that, might (pave occurred 

‘ ♦ * . J-i-S ’ * * ^ * * 

by placing the pairs for the consonants first, followed"- by those for the 

vowels. .* 

• * * 

k f . • * 

The first %C5 of t these paird'^ constituted Part-I'of the' test, and 
the next 4^, Part II. . A . , 



?: ) 






13 "Meaningful" here means "those that are acceptable as English word(s).". 
However, in the final test, there were two exceptions: .bon, con. \ 



d 

ERIC 



14 One pair (thaw : though) had two differences: voiceless : Voiced and 

Jp: p - (vowel sound as in <caught> for want .of a' better pair . . . " 



Ol 



^ further step In Par.t I was to replace a word occurring in ah. 
earlier pair, wi-th a word having the most ntlraber of' parajLlel features, 

, ' • a • 

but still^testincr the same element. * For example <^vole>, -oapiirred with <foal> 

• * % 

^Ln jaair number 30..- Pair number 45 was <vole : vole>, testing the /y/ bounds 

Thus, <vole> was replaced by <vote> and pair number 45 became <vote/vote>-. 

This step was taken to avoid any learning effect. - • . . 

.* • " : ■ . ; ■ 1 ■ 

The fprmat of Part II (example:" cab - cab/cap) did not allow “ 

, " ( ( . 
pairs of ’similar “words / lienee’, the list had to undergo the following 

: j 

.modifications before the final list was arrived at: . 

.* . * * * 1 ‘ | 

1(a) Wherever- a pair of similar words occurred, the I 

phoneme tested in that item was determined by 
reference^to the minimal, pale in v(hich the word . 
originally occurred, .and the similar pair then 
replaced by the minimal pair. • * 

4 .* (b) As was done- in Part. I, if any of the members of the 

minimal pair had , been used earlier^ tHe word(s) _ 

' was/were, replaced by a pair testing, tlje same' element. 

One- example of this process (l(a') and* (b)) .was item number 1 . 46’ which originally 

^ # H •. t q 

was lab/lab.' This was ‘replaced by the minimal pair lap/lab. Since this 

minimal pair 'occurred in Part I, it was replaced by cap/cab. • Sometimes 

'■ ■ • i . t _ £- ■ . 

■this process resulted- in •retaining’ only the phoneme to be" tested. Item 

.number 89’thigh/thy w^s .modified until it became thistle/this ’-11. This,, 
.incidentally, was the only example of using -more than one word as a, member 

q ^ •« * /f 

of a minimal pair. ... . 

2 . . Once all the minimal pairs were thus revised . ' 9 

and replaced,’ one member of the pair was , 

randomly chosen as the first stimulus. • 

•Ji A A ' 

Discussion of Parts I and II , ° 

The basis'- for establishing test items for Pyirt I and Part II can b' e ■ 
Questioned. The-reason for deliberately selecting the test items first and then 



‘ randomizing and including contrasts tHat were not in the. original list wa^ 

. * * . 
to ensure- that at least some significant contrasts like. 1 : r (lest :»rest) 

•(Perron, 1967); d : i (fode : bathe) (Lado„ 1961, 52) Were included. 

.Such contrasts, though they pose no problems to the native speaker, give 

/ •■ • . v , 

mych difficulty to a second language - learner, the diff iculty 'depending on - 

.) .■ ■■■ •■■•._/■' • 

/the sound system of h IS .language*. « . ■ . * 



A 



\. 



- \ 



Parts I and II had 45> items each, "as opposed .to less than- half 
tliat number in each of th'e other parts, because necessary pilot testing 
and item analysis had not yet been done. A second reason, the more important 
one’ theoretically, came from the nature of the study population, which was 
composed of at . least 30 ethnic groups. . While a particular sound contrast 
might prove difficult to one ethnic group, another*' group mighty find a different 

I ' * *♦ V 

contract to be more problematic. A large lil’t is more likely to include more 
- ' , - i: • : > 
representative problems than a short one. ' . - - . 

1 « r ■ \ » % . i 

technique • . ' , •... • • v • 

• / . " ■ • . * , 

. Lado (1961', pp. S'3-54) in his book "Language Testing'^says 'the 

• ' /• V ' ' ' • , , 

following of the technique used above: >. . 

- • "This is /an excellent technique for class- - , ( * . 

^ room- testing.- For formal tests, it suffer^ • 

' t from .tiie problem of all- two-choice tests : 

\ the effect of wild guessing ‘is quite heavy. . -. 

as for validity, it is more valid' thsya even 
extended observation of students in their . , , ; 

everyday use of the foreign language. ..In 
the. ordinary use of the language it is ’ 

V /difficult to know if the student has per- ; . , 

\ ceived the sound contrast/guessed at the ' 

meaning, or understood from the. context ' < 
rather than through the words containing the 
( difficult • sounds i In this technique, on the . 
other hand', .we are able to control th$ non- 
language factors and the language factors 
that are primarily pronunciation.” • • • , 

* l- t • ‘ ■* *. . 4>.«* . 

. (Laclo, 1961, pp. 53-54) 



o 



15 If we were testing the integrated skill rather than the elements ‘of - 
language separately, this, of course, would .not. be "an objection. 
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Part III 






Part III was comprised of 24 it'ems designed to test another aspect’ 

\ 



; 



I of auditory dis crimination — intonation. •' In English the same string of 

t ■ , - 

words can be said .in different ways to mean different things. The purpose 
here was to see whether the subjects could 'discriminate these differences. 

. t r ; . 

on hearing a given, utterance . • ( 

. The material was presented on tape. The subject. first heard an 

utterance (primary stimulus). After^a\few seconds, he heard two.. mo^e utterances 

(secondary stimulus), which gave two possible interpretations of what he first 

j' ~ ' i ' 

heard. He was to indicate the’ correct choice by darkening. A or-B; example: .v 

* • , * 

I want to go home. % A.D Home is where I B.O It Is I Who want to go 



3 



want to go. 



home, not him. 



. Format 

In addition to hearing the two choices, th'e subject alTso saw ’them 

.on paper. This was -,'to ensure that the subjects were not being tested’ , 'on the 

\ .• ' 4 . ' ■' : ; •• ' • ' 7 

secondary stimulus rather than .on the primary stimulus. The point of this was 

\ V . ’ ? 

fio determine whether the subject could hear the primary stimulus correctly. 

( \«t\ ' V • 

Thus, it was- deemed necessary to give as many cues as possible to^ the subject 
to minimize his difficulty^ in understanding, the choices.' (Providing the 

.alternatives in a students' mother tongue was' not practical nor ..was it known 

. . • / ' <- - - . ■ 

how well -the concepts could be translated..) . - V ’ 

- M - 1 ' V 

. IThe decision to let the choices be both heard and rea'd was primarily 

_ .* ■ y 

^.determined by the analysis of results of pilot runs yxth grade /five students. 

V ) ; ' ■ 

Choices were presented to subjects in three different ways. One group . - i 



only heard 

heard them, 

’ \ 

diffeJence 



them, another only read them, and a third groupi both read' and 

* V . . A " 

' % . ■ ' 

A comparison of means showed a mild ' (not |!atistically Significant) 

'among the three treatments in' favour of "read and hear," when the 

' ■ ' • - V - . ' .. ' ' • $ 
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" overall groups, made -up of both native and New Canadian students were considered. 
However, there was no difference. when only the native Canadians, were compared. 

The differences were attributed to the. New Canadians. 

■ , '*> 

• * . * ^ . %• . 
At the theoretical level, Perren (1967, p. 24) feels that a subject 

i‘s "...listening with a purpose..." when he ".. ..sees’ the choice of response ■ . 

, before or Simultaneously ^he^ring the statement." Ausubel (*1964, p. 423) 

* ‘ ^ a , 

agrees that "Simultaneous reading support can -furnish the necessary cues 
for meaning...." , •*. 

Admittedly, seeing the choice at’ the same time as it is heard puts 

* % ' " •. § • 

the. good reader at an advantage, “both over ^he poor reader and the non-reacler. 
However, it is likely that even a poor reader can pick up certain cues, 

especially if he is also hearing the words. Such a person may get information 

- e * •,( . ’ ’ * s ' 

from either or both hearing and seeing. This would 4 B,ut.o'nly the non-reader 

• V . . • ‘" A 1> . • " ,0 

at a great disadvantage. v It was assumed that,, the study population would' have— , 

: ■ • § v ‘- - • •’ ■ 

• a large percentage of readers, both good and bad, as opposed to* non-readers. 

The only other practical, .way to ensure maximum cueing was .by 

• ' ‘ 

repeating the'cho ices-: - ~ijb. .was observed during the pilot, runs with this treatment 

-• 7 * “'’■** ■ . * 

that- the subjects were very restive and bored, since there was too> much’ idle * 
time, particularly when both the primary stimulus and* the secondary stimulus 
were Repeated. A repetition of only the latter would have led io cognitive 



|UJ 

dissonance, since by the' time the subject finished hearing the choices the 



.16 



^second time, the primary stimulus would no longer be "ringing in their ears-. 

.. • : , - ■ • . \ r ■ - ■ 

Furthermore, repetition would have increased the artificiality of the linguistic 



.situation.' 



r» • 
1/ 



16 See Hutchinson in Valdman (1966, -p. 225)- for .a discussion. 



. . r 

Two-Choice vs. Multiple Choice ' r . ' 

/ V * V '• 

The on^p^Jlosti’f ication for having a two-choice rather than a 



multiple choice >test was the difficulty of getting. more than one good 
d is true tor . Thus , Part III -suffers from^tj’ie problems o‘f all two-choice 

b . . • . * • . 

tests, specifically the fact that the subject should be able to score 50% 
by guess-work alone. 



Voice 









( 



Unlike Parts I and II, this part of the test was recorded in 



different voices, to get the effect of Standard Variety. 

-• . . "Standard Variety means .any variety . * f 1 * 

spoken’ by educated speakers of that J 

language from any of the regions wheep- . . 

it is spoken natively."’ • • • 

(Lado, 1961 j p.-47), • • 

A Toronto student is bound - to hear a variety of English spoken' by his teachers 
and peers. To approximate the Standard Variety, five* voices from different 
parts of Canada, one adult female and four adult males, were used to. record 
• the primary .stimulus . Voices were randomly assigned to each test item. 

Tjse secondary stimulus (the* two choices* ^jor each item), however,- 
was consistently presented in the same, adult male voice. 

V ’ • ~ # 

Basis of Item Selection - . ’ • •: ' 

Gleason (l96l, Chapter 4), Hockett (1^58, Chapter 4^ and Long ’ 

» ' . . • * A ' • 

(19,61, Chanter 20) served as the basis for the items in Part III. The' 

• ' , _ ' ■ ' 4 * ' ' V . 

initial list consisted of 33 items, nine of-which were deleted from the 
final list, as being non-discriminato’ry#^>’the basis of pilot runs. These. •; 
were primarily items, wjiich 50% or more pf. the high scoring students got 
wrong or items ,on w6ich. the low scoring students did as well as the 
high scoring^students . ■ ; .. . • ■ < , 



/ 
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• The /items finally Selected were placed in order of difficulty, 

in an attempt to lessen anxiety that might be associated with this nbvel 

; . • 

test procedure. • 



Part IV 



' / 



Both Pa|ts IV and V tested vocabulary. Here again the words were 
limited- to the first 500 words of the Lorge-Thorndike count. Part IV ya& 

* t 

& 'test oh contentives and incliided six words each of which have more than 

V . m - , , 

■-> ♦ T 

qpe meaning. “"The testees were, given -twelve items which gave meanings of the 

$ * V , . ' 

six. words . • . . . 4 ' 

• A _ B 

Example -- part -of a play _ A. head . , 

2. a bill presented B., act* 

. * in Parliament 

. 1 } - ' 

3., a '-part of the body. 

above the neck - . 



4- to. do something 



Format 



This part was a paper ’and pencil test, and took the form of two- 
lists, A‘ and B. List B gave the test words, and list A the meanings. ‘I'his 
reversed order was determined by the requirements of the optical scanning 
machine used to score the answer sheets'. A change in the order would have 

.w m # 

resulted in the answer sheet having twelve choices. .Thfe subject thus fj^rst* 
read the meaning, and then found the word that fits the meaning. • .. 

Out of the six test items, two had three meanings each, two others 
had two meanings each, and- the other two had 'only one' meaning each. . , 

Basis fo^ Selection , * 

: Since the interest was in words which had multiple meaftings (and' 

there were not too many in the list of 500 words) initially only seven items 

■ ; • *'• :. •' } ' • * 

were picked. The Concise Oxford Dictionary provided a list of 21 meanings 

'.//"■ :■ "• / ;-/• . : V > '• 

-- - . . ■ ■■' , • ■ 

• • ••• » ■' / t ; ; . 



r 



~ that could be -expressed using the basic 500-words. Items iri both A and * . , 

J ' * ** 

B which, on the basis of pilot runs were either .too. easy ,or. too difficult, 
were dropped. The»final test therefore, had’ six test items and twelve 'chpices 
for meaning. 

• Part V " . 



- The difference between Part V and Part IV was that although both .-" 

•test vocabulary, Part V tested functors. Functors are not only- one of the 

. • 

, basic features of grammatical 'signalling in modern English, but pose 

’ t ,, * 

particularly difficult problems for- both the learner of English as a second 
- . - * - ' ) 

language and, the child from a low .socio-economic level. ’Schutz and Keis-ler 

. i ! 

(1968, p. 213} report that their results ".. .support the findings of other 

' * . 

investigators’ that, lower class children hre most deficient with functors" 

(as - compared with nouns and verbs). • 

M 

The subjects were given four functors and twelve choices, each 

( : ■ " ■ ' • . ' , 
of which .was a complete sentence when the blank . was filled in with a functor. 

A .. . ^ * * • / 

. Example -- It's time bed&r A. for 

• ' ' ’ B. after 

* t r « 

\ C. out 

■ ■■•’ . : , - - D. along 

Format 

■ v 

X 

• The format was similar to Part IV. One functor was 'the correct 

I * # V 

I , - 

answer four times ; v two functors wfere the correct choice three times; and, 

'the other two-were used twice. * 1 , * • . 

Basis of.rltem Selection ' • 

. * 1 * y 

• The criteria determining the selection, of itetas in this section 

, ; ; . • . . • . .- ...» 

were, again:, (a) test, words were to be from the Lor ge-Thornd ike list; *• 

* . * • * ^ 

• v ; (b)- sentences in which the words • occurred were als t o tp be.* 

, made up of words -from the^list. .. . /; >. ’ ‘ 
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An attempt (based on common knowledge) was also made to ensure that the 

*• , . * . '* .C\ 

concepts in the choice items were as "common" as possible. . 

•> * ■ ; . 

* These criteria initially provided a list of six test items and 

• *S * * 

22 choices, which was reduced to four test items' and twelve choi'ces, after : 

elimination of the easiest and the most difficult ones on the ‘basis of- ' 

^ • A ... 

\ 

pilot runs . , ' • . 

: ■ ■ : 

, ’ Part VI - ' . 


: ft . _ ( ; ■ ■ 


Part VI consisted of ten items .designed? to test- idiomatic usage. 

• * • . 

Idioms are different from vocabulary items in that the combination of a . . 

1 * " * 

particular sequence of words, carries a meaning different from the meshing 

* .• k' 

of the individual words that make up the idiom." English ' employs' this 
technique of idiomatic usage heavily, and therefore a grasp of this element 

* . * f; * , 

is an important^skill . ... 

In -this section, which was also caper and pencil, possible meanings 

. . v * ... . v. ; . 

of each idiom were provided. The rigorous .requirement of' limiting the 

vocabulary in the choices to the Lorge-Thorndike list had to be given, up here . 

for stylistic as well as other reasons. It was simply not possible to make up „■ v 

four choices for ten items within, the ' range' of 500 words. However, an attempt 

was made to keep "outside" words at a minimum. Aga'in, the choice was indicated t / . 

! f * 

'by darkening A, B; C or D. , ' . • • 

Example - What is the meaning of . . . '■ */ 

T. I haven't seen him for ages. A. L-havenit seen him / . 

• . for some time. 

B. I haven't seen him 
• since *he was 'four. 

' ' ♦ C. I haven't. seen him . 

' for a few hours. 


" • i 

-'W- 

’v o 


, . • D. I haven't seen him . ‘ . 

k • since' the meeting ^ j, v y 

. ; ,• • at which he spoke i 

" •' . ■ p ■ for 'ages . 
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Basis "'for Item Selection 



■ Test items were invented as- ideas occurred since a rood list of 

• ' , ■ * ■ ■ 

idioms could not be located. Whatever the biases in this process, in the 

. . * ' ■ ‘ . 

joilot runs, ten of the twelve items originally selected were found to be. 

discriminatory, and were retained in the test. Again, the items were placed 

'in order of difficulty. The four answer* choices for each item were randomly 
* » . * • • 

ordered . . 




CHAPTER,IV‘, — A PRELIMINARY EVALUATION 

• * • . > 

•" >* • /«> 

- \ ; 

Withia six months of beginning to look for a way of assessing 
. * ' • " 3 • 
a person's use of English, the materials, in their 'present form, wore prepared 

for administration to an expected student population of 6,000. The shortage 

r‘ of ^ ime is n °t an^excuse, but it limited the amount of literature that coula 

be reviewed in advance and the amount of material that could .be developed.- 

Pilot work with materials was minimal and item analysesawill. wait until it is 

determined whether there is any continuing need for the test materials. 

* \ • 

A Shortcomings 

Where 's English- Structure ? . ■ * . 

* - ' • * * 

Urammatical structure constitutes an essential sub-system- of language. 

i < * \ 

Any language ’test which does not -have a section requiring this knowledge** 

■ * . * , v- *•“ 

.should be considered thoroughly unsatisfactory. On these grounds alone the 
materials must be deemed incomplete, for they do not have items on structure. 
Strategies for such a section had been outlined} however, there was not time 
to develop these strategies. 

,"_I don't understand vour directions." . ' ■ 

. 0ne area requiring father research deals with .the language in which 

directions were given,. If the purpose is. to test the. student's knowledge of 
English, it must be done with carefully selected test items and not with 
standard directions. Otherwise, it becomes an exercise in understanding 
directions.. A child .whose English competence is poor would be unable to understand 
instructions in English. A solution is to give instructions. in his mother tongue. 
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In the present study with over 5,000 st\ldents whose first • 

; 0) « 

languages represented eighteen or mpre linguistic groups, such a - 

procedure uau ainiply out of the question. A.; the -too t has tape recorded 
• " * ; r 

instructions there is a possibility in the future of preparing separate ’ 

: tapes' with the instructions in different languages. This area (giving 

A %, 

% 

‘ directions in a language other than the language of the test) has not, 

to the best of this writer's knowledge, drawn' the attention of researchers. 

- The effects of giving directions in the subject's mother 1 tongue requires 

research. Certainly any test translations ought to be standardized. The * • . 

% ' * ' . . * ' 1 +P ' r g 

onus of translation should not be on the individual school that administers • * 


. 3 * ; • - 


the test. It must come in the package. „ 

Informal feedback from the schools indicated that the taped 


\ ' /• 


instructions were difficult. As a result of pilot work, the taped instruc- \ . 

i • • 

tions were completely revised twice. The major difficulty is not yet 

known; however, the medium itself L s suspect. Tapecl tests are so novel 

that it is quite conceivable that they interfere with the mental set for' 

* . ■ 
test taking which.North American students have developed. Actually, the % 

^ „ , . . . • . s . 

major complaint was that the taped material took far longer to give than 

the actual playing tim* of ' the .tapes, would' suggest. Preliminary examination • ’ 

of a handful of answer sheets suggested that once the students were working 


• . 




<. ' 


with the. test material from the tape they had relatively few problems. 

. - The Time-clock is North American! 

North American culture is a strongly time-oriented one. The, > >■ ■ > 

' Canadian-born child who has been reared in such a culture find's no problem 


t\ : ■ ’/ 


or at least no surprise - — in ^cing a test which has a time limit. 

However, the New Canadian subjects .of our study may riot have been . ■ 


■ • ■■ 


necessarily time-oriented, ' particularly if they ipame from a non-industrial ..3 ' 

. . . . . \ . . •. ■ - jV '• - ' \ 
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' \ ■ ; . '* ^ ^ • ‘ ■ . . ' . i 

background. -Undep such circumstances a speed test may have evoked anxiety * 

in the subject which could seriously impede his test performance. (Students 

from certain Canadian sub-cultures are* also peralized'by tests that insist ~~ 


, c ' . ' * 


•5* ” 4 | 

v oil speed.) • - • ~ - 


r 


Thus, it is' suggested that the E.C.T, be made a "po.wer-test, " T * 

v * . 

in which subjects are given as much time as they want to answer. Of course, 

• ‘ • * & « 
nothing in. the content need be changed for this purpose, only no time 


t 


should be stipulated. This could be done if the test were administered" . 

1 * ■ , V v * ' ' 

on an individual basis. * * * 

: . ‘ t 1 ; ^ % 

• , c * „ 

The effect of timing on performance also requires study.. The s'- 


' k 

% 

9 

■ « ' 


question’’ is particularly relevant to the recorded part of the test: what- * 

ever the cultural background,- it may be best to respond while-the stimulus 

* > 

is still ringing in the ear.' r. 

How Valid is the Word List? . , 

The entire E.C.T. , with .the exception, of a few items, was’ . | . 

» '• . • ' 

based on the Lorge-Thorndike list of the first 500 words (gee pc. \1 12).. 

. . ■ ’ ' , ' ‘ ’ - \ . . . • 

A better list of words would be one based on a representative -sample of * 

, # * f * 

■native Canadian fifth-graders, drawn from varied ethnic and sociB-cultural 

» ' • f 

groups. Fifth-graders are suggested because Grade 5 was the lqwest grade 
' level in the study. To make the E.C.T. applicable to lower grades that ( 

particular age -group should be sampled to ensure that the words used are 
within the students’ grasp. > . ■’* 

Another advantage in, working from^a new word list vrould be that this , - 

would permit the use of the first thousand words-, without seriously under- 


■ < 


mining understanding. With the Lorge-Thorndike List, . it was with less - 


■ • • r* . . . />;’ 


certailnty that additional words could be chosen from the second 500 wq.rds, , 


► «■ - J *’ ■ , 
S'i> 

li \ ■ 

• . ■£ . . . 


since the^list is outdated. The use of 1 ,000 words would be a valuable 

advantage that would enable the secondary stimulus (particularly in Parts III ; n f 


ERJC 


and VI) to be worded lgss awkwardly. ; ‘ 



i 
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t * Words Mot in First 500 of List . The, original- intent ion of 

restricting every word in the materials to the first 500 words could not • • 

be implemented completely because of problems in wording. Thus; in Parts III 

- - 4* 

and VI,. the secondary stimulus, was n"ot always restricted to the list, although 

an attempt was made to ensure that,. such was the case in Parts IV and V. 

V 

It must be reported that a few time,s words from the list of 
501 - 1000 words were used and even a few pother words . Some of these .words 
are given below: 

* ‘ ’ " * «r' * * 

* doubt, hen, chipmunk, cat,' cupboard, spent, Communist; ‘ ‘ , ■'? 

un-, -iAg, -ed were some other" units that were used with words 

^'from the list. The words outside the list werd considered to be familiar 

enough in the day-to-day life of the native Canadian subject. 

Your Bias is Showing! 4 • ■ ■ • • . * ' 

* * 
A final 'shortcoming was that test items were deliberately" and 

specifically chosen and thus may be biased in some way. Although th$ 

instruments' validity can be questioned on these grounds, such selectivity, 

was necessary. Lists do not exist from which the material for the jpartiGular 

t component parts could be randomly drawn. ' Such a random selection would not 
^ m * * # 

« . ‘ £ J . •• 

ensure the inclusion of all imnortant linguistic problems. The selection 

• . *■ ■ * ; . . - r ' v .- 

' * 

•still does not adequately recognize the .view >that certain linguistic sounds 
and combinations carry more meaning and functional^ load ^han others. 

, One particular problem of ' y,em selection in' language tests was 
pointed out by Lado (1950, p. 79)-: • . 

: . . •' j ■ ■ "\ - ... 

"Trying tests on native speakers of a langu-- 
1 ' age will/show some of the things that should 

be left -out; of the tests, but it does not 
tell us whax we should put into them." 
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Difficulties in the selection of phonemic .contrasts p re flown ’ 

* ' : ■ / 

h by Perren. (1967, p. 25),: - 9 . - 


• 

\ ' I 


} • • v ■ * * ■ . 

•■■■■. .’’...data is available in thfe relative frequency of . J ‘ 

’ occurrence of phonenics in particular varieties of .. 

' English., .but these provide no certain- indication^. - ' . 

of the relative functional load which they carry ^ ; , 

1 '■ ...selection of significant contrasts become part- . * 1 

icularly difficult when /test. . .has to be designed . - 

’ ‘ * for application to students with varying mother 

tongues." . 

. * . * 

» ■ vt t •' . •' 

Future Use * ‘ ' 


• 

t 

* 

t- 

, * 
• 


i ■ . . • 

Toronto has an extensive programme ‘for Mew Canadians of instruction 

, m :’s * / • • 

■h ' , A ' " 

in English as a second language. To the best of this writer's knowledge,* 
however, there is no (standard or non-standard) test used to determine the l 
students' level of competence in "English. Probably none suits the schools ' 

t ♦ ’ % 


X 


' * requirements since most language tests are des igned>f or students, who already 

have a fairly high level of English competence. 

.One value of the EiC.'F. lies in its potential use with students 
. r - r , 

who arrive with a little knowledge of English. Since half of the Test was 

♦ •• - * # 

auditory, even those students with no writing or reading ability in English 

* . . * - j ‘ 

* could be tested. Further, the Test was construpted on a alTidiag scale of 

* » 

> , ■, * v : , 

difficulty. 'The Mew Canadian who is able to answer ’only ' Parts- i and II \ 




* (phonemic contrasts) possesses lessSfinglish competence than one who can ' 

•. * q , ■*' 

answer Part III (intonation) . A student who correctly answer? Part VI 


k * ■ 1 . 


(idioms) is certainly more advanced than the .gne who stbps at Parts IV and V 




(vocabulary). I£ should be reiterated here that if -such use of the instrument • 

*' : ; ■-> - ' ; . • 

were to be. mhde,’ the directions should be given in the native "language of, 


■; . .. ■ j 1 -. ; . . 


the- student, q' \ * " • 


> • .. 

■; ■j;;' 




■ T ■ 


;• 1?- Data are available on the functional' load of the commonest words . 


» • ' . f v ' ' . 

C" ■ o 
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{See, for example, Lorge’s U949J semantic count.; . 



I 



The test v may eventually be useful for several purposes, including 

programme placement, diagnosis and ev^n possibly as a measure of achieve- 

■" • * . • *\ * ■ 
ment. These possible applications would require rformative data and also 

modifications and extension of the materials. . ‘ , 

^ ' *• • . . 

• • • » ■ ■ 

-■ . Closing Exercises v ‘ ' .* 

■; * • . • 

N The E.c.T. is far from perfection. I.t was an attempt to solve 

% t • : . * " > 

% # 

some- of the problems 'of testing English an a second language. The major 

l ' 

contribution, it , is felt, was the break from th>~traditional patterns used 

* * 

in ciprent language tests. The vocabulary was limited to the most frequently 

used ’words. The testing Situation could be managed ‘by a’ student with a 

^ . 

limited number of English wofds at his command. -€t is hoped that these tests 
* ■ , ' \ 
of language will be uncorrelated with'tfiptelligence. * 

A second contri^u't^ion may be the shift of emphasis to testing 

language skills, without reference to the classroom. Answers about the value 

of the' test and further relevant data will follow. An analysis of the results 

of the E.C.T. as administered to the more than 5 » 000 students as part of the • 

IB P J 

New Canadian Study is planned as a separate report . ' • • 



18 ^The, materials used in the E.C.T. x^ill be made available by the Research 
i Department on special request. » . 
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List of English Consonant Phonemes as. Given by Gleason! ( 1961 . p. 




(See Footnote 6 ) 

S° f 



291 

\ 



the underlined go&id . in 



<cat> 



'i 









<oin> 

<bin> 

<tin^ 

. \ 

<din> 

<kin> 

< gong > 

<check> 

<jeep> 

<fin> 

<victory> 

<thin> 

<this> 

<sin> 

"\ <zone> 

,<shun> 

•<vision> 

* <man> 

* x 

<man> 

‘ <sing> *’ ( 

<live> 

<ring> 

<win> 

<young> 

<hat> 

. .. ' . v 

> 

. « •• . 

Although most of the examples above ha”ve : been in initial 
position, phonemes occur in initial, medial and final positions, with a 
few exceptions . For example, /i/ doesn't occur initially. 
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Recommended References 

^ * 

I . . i " . . 

A - . Bo6ks ‘ . • ’ ■“ 

The following three books on Linguistics have been 

*\ ** f 

written for the layman. These bobks present the material in 
a non-technical and attractive manner: ' * ’ 

Hall, R. A., Jr. Linguistics and your language . 

(Revised edition of Leave your language alone . ) 
New York: Anchor Books, Doubleday & Co., I960. 

Sapir, Edward. Language . New York: Harcourt, Brace 
\ & World, 1921. 

'/ Scar gill, M. H.., & Penner, P'; g/' (Editors) Looking 

. ;■ at language . Toronto: W. J. Gage, Ltd., 1966. • 



B. Periodicals and Journals 

■ English'- A New Language . 



Commonwealth Office of 



'^’•Education, Sydney, N.S.W., Australia. 

, _ "" * «# 

^English' Langua ge Teaching . The British Council, 

65 Davis Street, London, England. 

English' Teaching -Abstracts . The British Council, 

65 Davis Street,' London, England. 

English Teaching -News . The Britisjj Council, 

65 Davis Street', London, England. 1 

Language Learning: A Journal of Applied Linguistics 

1522 Rackham Building, Ann Arbor, Michigan/ 

Linguistic Reporter . Center for Applied Linguistics, 
. 1755 Massachusetts Avenue, Washington', D.C. - 

M. L. Abstracts . Fullerton, California. • 

The Mo"dern Language Journal . Curtis Reed Plaza, 

Menasha, Wisconsin. - 

N. A.F. S.A. Newsletter . National Association of 

Foreign Students Advisers, 500 Riverside Drive, 
New York 22, New York. / * • 



^ List from Finocchiaro, 1964, pp. 140-141. • . 
*# Not in Finocchiaro' s list; 



